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students' reactions to the electronic worksheets they encountered in their 
first 3 sessions on the ILS. Each session was video-taped and the students 
were individually interviewed at the end of the three sessions to determine: 
(a) what new procedures they were required to learn in the computer 
environment, and (b) the understanding they gained of the content delivered 
to them via the electronic worksheets. This paper reports on this interview 
which focused on how students "got to know" the procedures which would enable 
them to progress through the ILS. The results showed that the students: (a) 

had to adjust their "school maths" procedures to accommodate the ILS 
procedures, and (b) were able to progress in terms of the systems' evaluation 
of their ability with impoverished understanding. The paper discusses reasons 
for this and the propensity of systems of this type to focus on syntactical 
and instrumental understanding. (Author) 
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An integrated learning system (ILS) is a computer-based tutoring program which provides students with learning 
experiences in many disciplines across many years of school. This paper reports on two Year 6 students' reactions 
‘to the electronic worksheets they encountered in their first 3 sessions on the ILS. Each session was video-taped 
and the students were individually interviewed at the end of the three sessions to determine: (a) what new 
procedures they were required to learn in the computer environment, and (b) the understanding they gained of the 
{ content delivered to them via the electronic worksheets. This paper reports on this interview which focused on 
how students "got to know" the procedures which would enable them to progress through the ILS. The results 
showed that the students: (a) had to adjust their "school maths" procedures to accommodate the ILS procedures, 
and (b) were able to progress in terms of the systems’ evaluation of their ability with impoverished understanding. 
The paper discusses reasons for this and the propensity of systems of this type to focus on syntactical and 
instrumental understanding. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



The Integrated Learning System 



One of the by-products of the growth of information technology in education has been the computer-based integrated 
learning system (ILS) which includes extensive courseware plus management software. An ILS has three essential 
components, namely, substantial course content , aggregated learner record system , and a management system 
which "will update student records, interpret learner responses to the task in hand and provide performance feedback to the 
learner and teacher" (Underwood, Cavendish, Dowling, Fogelman, & Lawson, 1996, p. 33). 



The ILS used by the students in this study was a comprehensive instructional system powerful enough to deliver complex 
courses to students from pre-school to adult. According to the manufacturer, its courses were designed to foster the 
development of foundation skills and concepts and to promote the use of higher-order thinking skills, therefore 
encompassing skill/concept development, problem solving and investigation. 



It should be noted that the manufacturers endorse the system only as a tool for teachers to use to consolidate already 
introduced material and to diagnose student difficulties with this material. They argued that it is the teachers' role to 
introduce the material to be practised on the ILS, and to remediate the difficulties identified by the ILS. Therefore, they 
contend that the effectiveness of the ILS depends on the quality of teacher input and consequently any evaluation of the 
ILS should take into account the role of the teacher in relation to the program. 



The ILS in this study was a closed system, that is, the curriculum content and the learning sequences were not designed to 
be changed or added to by either the tutor or the learner (Underwood et al., 1996). Its major feature was its management 
system which, according to the manufacturer, has three main functions: (a) to deliver courses to each student according to 
the teacher's instruction; (b) to manage all student enrolment and performance data; and (c) to provide the means for 
teachers, laboratory managers, and administrators to organise the use of the courses, and to monitor student progress. 



Numeracy within the ILS : The ILS core numeracy course is divided into a range of topics (e.g., numeration, 
addition, multiplication, fractions, space) which are then sub-divided into collections of tasks that are sequenced in terms of 
performance at different levels . The difference between levels was constructed so that high mastery at one level 
(approximately 85%) is the same as mastery (above 60%) at the next level. The core numeracy course is based on USA 
syllabi but correlates reasonably well with Australian syllabus requirements; individual tasks were developed and placed in 
levels as a result of large-scale trials in the US. For their initial placement on the ILS, students are given a large number of 
tasks at different levels until the system finds the level at which they have reasonable mastery (about 65-75%). When 
students achieve high mastery at one level, the system automatically raises them to the next level. To maximise the chance 
that task performance correctly represents level, the tasks within a level are presented randomly. Any reduction of 
randomness affects the accuracy of placement and, therefore, the potential for students to achieve mastery. Without 
mastery, students may not experience the continual success, and therefore the motivation for achievement, that lies at the 
theoretical heart of the ILS. 
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The ILS tasks are in the form of electronic worksheets which are generally attractive in their presentation and sometimes 
creative in the way they probe understanding. They attempt to encourage the construction of knowledge by providing 2-D 
representations of appropriate teaching materials in mathematics (e.g., Multi-base Arithmetic Blocks, Place Value Charts, 
fraction and decimal diagrams). 

Built into the core numeracy course are online student resources that enable students to get special help during a session 
should the need arise. For example, the Help icon provides the answer to a task whilst the Tutorial icon provides 
student-friendly information on how to do a task and the thinking required for its solution. (For some questions, a repeated 
error triggers the provision of assistance and hints.) However, use of the Help and Tutorial icons automatically grades 
performance as incorrect. The Toolbox icon makes calculators, rulers, tape measures and protractors available for student 
use and also provides complex tools (e.g., graphing and drawing) for advanced levels. The Math Reference icon provides 
definitions of words, sets of tables, and sets of currency from a variety of countries. For lower levels, there is an Audio 
icon that reads text passages to the students through earphones. 

Some tasks have novel presentation formats which students find difficult to interpret (e.g., spring scales used to determine 
number size, not object mass). Other tasks require inflexible and/or novel solution formats which result in students' correct 
answers being marked incorrect (e.g., failing to type the units digit first in operations) as are responses which differ 
syntactically from the expected responses even when they represent semantic understanding (e.g., the omission of zero in 
decimal numbers such as 0.63). There is a tendency for questions to be closed (i.e., "find the right number") and a 
tendency to base performance on speed (although the teacher can vary the time limits on answers). Time delays (e.g., while 
an algorithm is completed with pen and paper) can lead to the ILS's defaulting to incorrect. For each level and topic area, 
there are worksheets that can be printed thus providing students with extra practice and teachers with a guide to the types 
of activities that need consolidating. 

The ILS is designed to work with individual students, a feature which appears to be contrary to modem teaching/leaming 
principles which encourage group work that promote verbal and higher-order reasoning skills. However, students normally 
work on the ILS in mathematics for a maximum of 45 minutes per week over three sessions so the amount of time 
working alone is small when compared against the total weekly time spent on mathematics (usually 200-225 minutes). 

Effectiveness of the ILS : Few evaluations of the ILS used by the students in this study have been published in 
reputable journals. A computer search found only two (other than the authors'), the first was a review and synthesis of 
evaluation reports of integrated learning systems in the elementary and primary grades by Becker (1992). This review was 
critical of many evaluations of integrated learning systems per se , especially those reported by manufacturers. Becker 
found that many evaluations had sample bias, inadequate data collection, poor data analysis, inadequate description of 
implementation, and inappropriate or non-existing controls, making it difficult to draw strong conclusions about the 
existing data. He re-evaluated 32 elementary and middle-grade studies which met criteria of having more than 50 students 
and the existence of control groups (for all but 3 studies). Becker re-analysed data in terms of effect size (ES), a measure 
of the difference between trial and control students "expressed as a proportion of one standard deviation of individual 
performance scores" (p. 9) in the control group. He argued that educationally significant differences would be +0.3 for a 
one-year trial and +0.47 for a three-year trial. His conclusions were that only manufacturer-based studies reported 
significant gains (ES>+0.3) for the ILS. However, these studies, had "used the unusual procedure of eliminating cases 
from data analysis that showed sharp declines" (p. 30) while retaining cases that showed large gains. Independent 
evaluations showed negligible to moderate gains (0<ES<+0.3). 

The second evaluation (Underwood et al, 1996) was a study of the ILS's effectiveness over 6 months in four primary and 
five secondary schools in Great Britain (251 students in the trial and 131 in the control) by). Using Becker's ES method, 
Underwood et al. found a substantial positive ES (+0.4) for mathematics performance. For primary students, trial students' 
performance was significantly superior to control students in addition, subtraction, multiplication and extensions. However, 
there were only 20 trial and 19 control students in the primary mathematics evaluation, which would have excluded it from 
Becker's (1992) analysis. The numbers for secondary were larger (trial -77; control - 57) and significant gains were made 
in the domains of operation laws and diagrams. In neither the trial nor the control group was there a significant positive ES 
for fractions (which is of interest to this study). 

The evaluations reported in Becker and Underwood et al. used standardised tests to compare mathematics knowledge 
gains. However, standardised tests do not provide explicit information on students' mathematical knowledge structure and 
thinking strategies. 



History repeats itself 

Integrated learning systems are reminiscent of the Individually Prescribed Instructional (DPI) packages that proliferated in 
the US in the 70s with the ILS activities presented in electronic, rather than paper, form. Both systems have a management 
system which marks students' responses, directs unsuccessful students to other similar activities until "learning" 
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(familiarity?) takes place, and directs successful students to another higher level, and the process is repeated continuously. 
The only real difference between the two systems is that, in any ILS session, activities cover a variety of mathematical 
topics whereas in any IPI session, activities are presented in finely detailed sequences within the one topic. The 
pedagogical flaws in IPI systems were exposed by Erlwanger (1975) when he undertook a series of interviews with a 
variety of students in an attempt to understand what mathematical knowledge students acquired from individualised 
instruction and how that knowledge was acquired. One student, Benny, had been perceived by his teachers to be "very 
good" at mathematics, a perception that had been gained entirely from his rapid rise through the levels of instruction. The 
interview with Benny revealed that he had constructed several misconceptions that enabled him to accommodate the variety 
of answers that were demanded by the discourse of the package. The following protocol provides an instance of Benny's 
misconception with respect to decimal fractions and his "incorrect generalizations about answers" (Erlwanger, 1975, p. 15). 

He had previously solved 2 + .8 = 1.0 (a revealed error pattern) and 2 + 8 / 10 = 2 8 / 10 and was explaining (unsolicited) how 
IPI's answer key [the ILS's marking system] would mark him if he interchanged the answers to the particular problems. 

Q 

B: Wait. I'll show you something. If I ever had this one (2 + .8) ... actually, if I put 2 °/jq, I get it wrong. Now down 

o 

here, if I had this example (2 + / 10 and I put 1.0, 1 get it wrong. But really they re the same, no matter what the 

key says. (p. 15 ) 



Context of the study 

This study comprises the second stage of a three-stage evaluation of the ILS. The first stage was funded by 
Education Queensland; the other two stages were funded by an ARC SPIRT grant. Stage 1 aimed to: (a) 
evaluate the ILS's effectiveness in enhancing numeracy and literacy (particularly for low-achieving students); 
and (b) identify factors that influenced teacher endorsement of the ILS. All Queensland schools (primary, 
secondary, and other special schools such as prisons) were invited to participate in the year-long trial. 
Education Queensland developed and administered pre- and posttests for numeracy and literacy; the QUT 
research team visited all schools at the beginning and end of the trial and selected focus schools in the middle 
of the trial. During each visit, students were observed as they worked through their ILS sessions, and all 
personnel connected with the trial, namely, teachers, students, administrative staff (principals, deputy 
principals, teacher aides, computer technicians) were interviewed. In all, approximately 1000 students and 45 
teachers from 23 schools were involved. 

During the visit to the focus schools, eight Year 6 students and eight Year 8 students (including those students 
who had made impressive gains according to the ILS reports) were given a diagnostic test related to the content 
they had mastered on the ILS and to their attitudes to the ILS; the diagnostic tests were developed by the QUT 
research team and administered in individual interviews which were audio-taped. The findings from the 
standardised testing are not yet available. However, the diagnostic interviews revealed that students generally 
had impoverished knowledge of the numeracy domains they had appeared to "master" (Baturo, Cooper, & 
McRobbie, 1998; 1999). With respect to attitude, students generally liked working on the computer ILS but, 
irrespective of whether they liked it or not, most students believed it was helping them learn. Teacher surveys 
and interviews indicated that their perceptions of the success of the ILS (i.e., students' performance as 
measured by their placement and progressions on the ILS) were related to the interaction of three sets of 
characteristics: (a) of the users (the students), (b) of the teachers; and (c) of the ILS (Baturo et al., 1998; 
Baturo, Cooper, McRobbie, & Kidman, 1999). This interaction is illustrated in Figure 1. 
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Figure 1 . Characteristics affecting performance ( placement and progression) on an 1LS. 



Stage 2 followed seven Year 6 and seven Year 8 students across numeracy sessions and another seven Year 6 and seven 
Year 8 students across literacy sessions in a 6-month trial. Neither the teachers nor the students had used the ILS before 
and, unlike Stage 1, supervision of the students on the ILS sessions was provided by the QUT research team. 

Stage 2 focused on students’ acquisition of knowledge in an ILS environment via persistent detailed observations of ILS 
solution procedures and regular individual diagnostic interviews to ascertain the quality of the knowledge constructed, 
namely , syntactic or semantic knowledge (Resnick et al. 1989). To make the interviews feasible, the ILS numeracy and 
literacy domains were restricted. By extending Stage 1 in this way, Stage 2 aimed to confirm and refine relationships 
between ILS performance (i.e., placement and progression) and student characteristics (as in Figure 1). In particular, the 
study examined whether: (a) initial placement was low in relation to existing student knowledge (see Kidman et al., 1999); 
and (b) progress, as measured by the ILS, reflected syntactic rather than semantic knowledge. This paper addresses the 
l at tpr aim of Stage 2 by focusing on two Year 6 students (1 high-achieving, 1 low-achieving in terms of the pretest). 

Stage 3 repeated Stage 2 but with a Year 6 and a Year 8 class that had been identified by the ILS manufacturers as the most 
successful users of their product in the Brisbane metropolitan area. In this study: (a) the teachers were experienced users 
of the ILS, (b) the ILS sessions were integrated into their existing programs, (c) there were no restrictions on the ILS 
domains, and (d) the diagnostic tests focused on students’ understanding of the ILS content they were working on at the 
time of the interviews. 



Methodology of Stage 2 

Subjects : This paper reports on the numeracy development of two Year 6 students (Lance, high-achieving; 
John, low-achieving) who were part of the larger study comprising 14 Year 6 students (7 numeracy; 7 literacy) 
from a small Brisbane inner-city state primary school and 14 Year 8 students (7 numeracy; 7 literacy) from a 
large Brisbane metropolitan state secondary school. Both schools had a range of students, but were 
predominantly middle class. The schools and teachers volunteered for the project and neither the teachers nor 
the students had used the ILS before. 

From the pretest, 2 high, 2 medium and 2 low-achieving students from each year level were selected for 
in-depth study in numeracy and in literacy. As well, a very high-achieving student from each class was 
selected to focus on the first objective, namely, to study whether the ILS placed students at a level that was 
lower than their known ability. The students rotated through three 15-minute ELS mathematics sessions per 
week on computers in their classroom. Because of the study’s restrictions, there was no integration of the ILS 
mathematics with normal classroom mathematics. 

Instruments: A pre- and posttest focusing on two crucial knowledge structures that need to be established in 
the middle years, namely fractions (common and decimal) and mn lti p l uM mty (<-» and I operations, and 
area) was developed and administered. A pre- and post-student questionnaire covering affects and beliefs with 
respect to numeracy/literacy and computers/the ILS was also developed and administered. Two semistructured 
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individual interviews were undertaken - one at the end of the placement period and one at the end of the trial 
(6 months). The first interview focused on the students' observed behaviours as they became familiar with the 
computer environment; the final interview focused on the students' understanding of the concepts and 
processes they had "learnt" during the computer sessions, and their feelings about learning in this 
environment. 

ILS reports provided records of each student's time on the computer, and gains (given in years) as measured 
by the management system. 

Procedure : A laboratory of 3 computers was established in a small withdrawal area at the back of the Year 6 
classroom; the Year 8 class used an already established laboratory of 20 computers. The ILS was programmed 
to provide only those tasks that related to fractions (common and decimal), multiplication and division 
operations, and area measurement. The students were rostered for 3 ILS sessions per week during school time, 
ensuring that the times did not clash with classroom lessons that students particularly enjoyed (e.g., art, sport, 
play time on the class's school computer). Video snapshots were taken at regular intervals of their ILS 
sessions. Students were withdrawn from the classroom and interviewed individually. The interviews were 
audiotaped. Notes were taken on a specially prepared interview response sheet. 

Analysis : The video tapes were viewed and instances of students' dilemmas as they got to know the ILS 
procedures were noted. These formed the basis of the items of the first interview and so each first interview was 
idiosyncratic and was contingent on the observed video behaviours. Both interview results were translated into 
protocols, behaviours categorised, the questionnaire results were coded and these results combined with the 
test results and ILS reports to provide a profile of each student. 

Results for Lance and John 

The ELS marked students wrong if they recorded solutions incorrectly, if they took so long that the ILS's hint 
process was activated, or if they themselves activated the TUTOR icon. Incorrect task performance was 
generally due to keyboard skills, ILS idiosyncrasies or from lack of understanding of the task. For example, 
most students encountered a task similar to the one in Figure 2. 

If students failed to answer quickly enough or if they answered incorrectly, a hint appeared below the number 
line (see Figure 3). 
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Figure 2 . An ILS task. 

Figure 3 . The previous ILS task with ILS-activated teaching hint. 

John had not been able to do this even when the hint has been provided. The following protocol (I = 
interviewer; J= John.) indicates that his problem was in identifying the symbols so, in his case, the hint was 
not helpful. (Two other students in the larger study revealed during their interviews that they knew what the 
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symbols meant but did not know how to produce them on the keyboard.) 

I: You seemed to have some problem with this task, John. [J: Yeah, I always get them mixed 
up .] Do you know now what they mean? [J: Sort of .]. How do you read that one (<)? 

J: What do you mean by "read" ? [I: Well, when you see that, what goes through your head?] 

It looks like a triangle . [So you don't know the words to put with it?] No . 

I: "Is smaller than" (writing below the symbol) is how we read that sign. So you didn't get that 
from the ILS? [Nah ] And that one (>), as you can imagine is? [Bigger than .] Yes, bigger or 
larger or greater. So, when you came to here, it wasn't that you couldn't compare them (the 
fractions), it was just that you didn't know what they (referring to the symbols) meant? 

[Yeah ] (John was then able to complete the task successfully.) 

Another problem related to entering data involved the use of "0" when writing decimal fractions (e.g., 0.7). In 
most tasks, the ILS omitted the 0 units but in some tasks it included the 0 units. Lances' protocol illustrates 
this initial confusion and how the students accommodated these recording isiosyncracies. 

L: When you first started to write, say 9 tenths, did you try to put zero point nine? 

L: Yeah, but it didn 't work! 

Problems related to ILS procedures 

Learning to use the ILS tools and how and where to record answers were the two most common procedural 
problems encountered by the students as the following protocols illustrate. 

I: Does the ILS do some things a different way from what you used to do in class? 

J: Yeah, and one thing I got so confused about, um, one of the sums, um, it would make you put in 
the other numbers first, like a calculator . 

I: Like instead of writing the number from left to right, it makes you write the number from 
right to left? [J: Yeah, and I'd usually get it wrong and I couldn't work out why .] 

This was a problem that beset almost every student in the larger study. For number facts (e.g., 7 <-» 4), the 
students had to write the 8 ones before writing the 2 tens; writing the 2 tens followed by the 8 ones was marked 
incorrect. However, in the following perimeter task, the student was expected to write the 2 tens before the 6 
ones. This inconsistency led to many errors and initial lowered performances. 

I: Have you learnt something new on the ILS that you haven't learnt in class yet? 

J: Yeah, I guess, um I used to be pretty bad at, um, ah (indicating a rectangle) ... [I: Area? ] 
Yeah . [I: Perimeter?] Yeah, but area mostly . 

John had encountered the task shown in Figure 4 in which he 

had to measure the breadth of the rectangle and then calculate its perimeter. 

I: You took a long time to get started on this task, John. 

J: Yeah, I had to add these two together and I didn 't understand. 

The lady (ILS supervisor) told me where the ruler was . 

The ruler can be activated through the Tools icon. However, there is a ruler in the vertical orientation and 
another in the horizontal orientation. Because of the orientation of the rectangle, John needed to activate the 
horizontal ruler but he tried to activate the vertical ruler to no avail. This added to his frustration with the 
task. The supervisor suggested that he click on the horizontal ruler which then enabled him to calculate the 
breadth of the rectangle (10 cm). However, he had had three unsuccessful attempts at calculating the perimeter. 
The following protocol probes John's thinking at the time. 

I: First you wrote, 6, then you wrote, 15, then 16. Do you know how to find the perimeter? 
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J: You add two of them together and then you add them all altogether . 

I: Could you add these two (3 cm and 10 cm) and multiply them by 2? 

J: Vi m, no all I do is add those two together (one length and one width) and get 13 and then add 
these two together (the other length and width) and there's 16 (adding the 13 to the 3 cm). 

I: If you add these two together (the other length and width), wouldn't you be getting 13 again? 

J: Yeahhh, oh, yeah - so 26 . 




i 

mjo 



Both Lance and John encountered problems in which tenths were to te added.' The symbolic recording 
required was supported by pictorial representations of the tenths to be added (see Figure 5). The first of these 
types did not require regrouping (e.g., .7 + 2); follow-up tasks of this types did require regrouping. Thus, the 
tasks were sequenced appropriately and the link between pictorial, language, and symbolic representation is 
universally recommended for facilitating understanding of concepts and processes. 

Figures . Adding tenths. 



When first faced with these tasks, both boys did not know how to proceed. John immediately asked for assistance from 
the TT -S supervisor. Lance tried to colour in the requisite number of tenths in the 1st box. When this didn't work, he then 
tried to take some tenths from the 2nd box to place in the 1st box. The supervisor intervened to show Lance that he had to 
click on a red tenth and the next tenth in the "blue" box was immediately coloured red. When the appropriate number of 
red tenths had been added to the blue tenths, the students was expected to click a button marked DONE. This then 

activated the symbolic representation of the operation, namely: .2 + =• The two answer sections seemed to confuse 

i nnrp and eve ntuall y the supervisor intervened. Lance was then able to complete this task successfully but not the next task 
of this type which required regrouping (add 9 red tenths to 2 blue tenths). He wrote .2 + .9 = 1 1 (omitting the decimal 
point). The following ILS hint was activated (see Figure 6). 
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Figure 6 . ILS hints given for task requiring addition of tenths. 

John had encountered an example which required adding 8 tenths and 8 tenths. He translated the ILS expression, .8 + , 

as .8 + IQ. The ILS immediately appeared with the following hint: You added eight tenths. Type eight tenths as a 

decimal . John did so and when presented with the ILS equation, .8 + .8 = , John wrote 16 (omitting the decimal point 

as Lance had done). The same hint as shown in Figure 5 was then activated. John wrote 16 as the number of tenths left 
over and again the ILS overwrite his incorrect response with the correct response. John eventually wrote the correct 
response (1.6). Like Lance, John was able to complete all other examples of this type correctly. 

To determine whether their subsequent correct responses were semantically based or syntactically based, the students were 

asked to complete a similar example (.8 + .5 = ) in the interview. Lance wrote 1.2 (his number facts were not well 

known) and his protocol suggests that he had learnt the procedures required by the ILS to "get the right answer" but had 
no understanding of the procedures. 

L: Point 8 plus point 5 would equal 1 point . . .point 2 (sounding unsure) [How did you get that?] 
Because if it was zero point - no, point zero eight plus point zero five, that would be, ah, hundredths 
so, um, so when you added it, you wouldn't need to go above the point - the decimal so, um, you 
could just put it up one more for the tenths . 

I: So you're saying that's where you got your 1 from (in 1.2)? [Yeah, well no, I got the 1 from - cos I 
added those two together ] And how much does that give you? [That gave me 12 - so it was 8 tenths 
and . . .] So the answer has to be 12 tenths, not just 12 ones? [Yeah ] 

To determine whether his lack of understanding emanated from the regrouping required, Lance was asked to complete an 
example in which no regrouping was required (.2 + .7 = ). 

1: So if you had 2 blue tenths, and you had to add on 7 red tenths, what would you put for this (writing .2 
+ =)? Lance wrote .9, anticipating the answer). 

Discussion, conclusions and implications 

Both students were enrolled at Level 5 for their first session. However, by the end of the 3rd session, there were already 
marked performance differences across all the mathematical domains. Figure 7 provides the students' progression rates 
across the 18 sessions within the decimal fraction domain to exemplify the relative performances within all mathematical 
domains involved. 
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Figure 7 . Progression rates for Lance and John within decimal fractions across the 18 sessions. 

The high-achieving student, Lance, progressed steadily across the sessions in all mathematics domains 
(see Figure 8) whereas the low-achieving student, John, "progressed" erratically (see Figure 9). 
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Figure 8 . Lance's (HA) steady progression within all domains across the 18 sessions. 
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Figure 9 . John's (LA) erratic progression within all domains across the 18 sessions. 



We believe that the differences between the students' ILS progression rates are a consequence not 

only of the students' knowledge and abilities but also of their affects , beliefs , computer knowledge , 

and the interaction of these user factors with the ILS factors listed in the ILS-mediated learning 
environment model (see Figure 1). 

The ILS was a closed system, providing random practice mathematics worksheets . Some worksheets 

had novel ways of representing information and were ambiguous with respect to what they required 

of the learner . The sessions showed that Lance was much better able to handle the novelty and 
ambiguities of the ILS worksheets . He was generally able to identify what was required for the 
various worksheet types. On the other hand, John was less able to come to terms with the 
requirements of the worksheets with the irregular results of Figures 8 and 9 as the consequence. 
Lance had more reasoning ability and more knowledge than John; he liked mathematics, enjoyed even 
the practice component of mathematics activity, was motivated to achieve and believed that the ILS 
worksheets were helping him to learn. John did not like or enjoy mathematics, was unmotivated and 
did not believe that the ILS was relevant to him. 



With respect to prior computer knowledge, the interviews revealed that both students had access to 

home computers and enjoyed their sessions on the school computer where they could do word 

processing and play games. From these experiences, Lance appeared to associate computers with both 
work and play but John appeared to have associated computers mainly with games. Thus, John 
entered the ILS trial with preconceived beliefs about the role of computers in learning that may have 
affected his motivation to overcome dilemmas arising from novel presentations. 

In the Stage 2 study, the teacher was deliberately removed from the ILS-mediated learning 

environment so that we could gauge more effectively the quality of the interaction between student 

and ILS. However, the Stage 1 study had shown that the teacher's role in the environment was 
pivotal. If teachers believed that their student would benefit from the ILS, then they were more likely 

to integrate the ILS results into their classroom mathematics lessons and establish a system of 

rewards to promote motivation to continue. These teacher behaviours tended to promote the students' 
beliefs in the value of the ILS. The Stage 1 study indicated that without teacher support, students 
soon grew to dislike the ILS sessions intensely. This phenomenon occurred in the Stage 2 study. All 
14 students had to be withdrawn from the ILS before the 6 months was completed. 

The experience of Lance and John show that integrated learning systems such as the ILS have 
difficulty operating without teacher support and integration into other classroom activities, even though 
they have the capacity to do so. The experience also shows that less able students, often a target 

group for promotion of an ILS, may have more difficulty coping than more able students. The 
operating systems of integrated learning systems built around prototypical tasks and inflexible marking 
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require reasonable knowledge , motivation, beliefs and reasoning abilities for students to persist without 
support. 
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